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1. Introduction 



Mean field models in statistical mechanics furnish nice examples for the interpretation of 
thermodynamics as the theory of large deviation for Gibbs measures of microscopically defined 
statistical mechanics systems [E]. Roughly speaking, in such models the Hamiltonian is only a 
function of (extensive) 'macroscopic' quantities (density, magnetization, etc.) of the system. In 
the thermodynamic limit, the distribution of these quantities is expected to be concentrated on a 
sharp value and to satisfy a large deviation principle. The corresponding rate functions are then 
the thermodynamic potentials (free energy, pressure) that govern the macroscopic response of the 
system to external (intensive) conditions. The classical paradigm of the theory is that the number 
of relevant macroscopic variables is excessively small (order of 10) compared to the number of 
microscopic variables (order of 10^^) . 

Over the last decade, the formalism of statistical mechanics and thermodynamics has found 
increasing applications in systems in which the macroscopic behaviour is far more complex and 
described by a 'large' number of variables. Such systems can be found in biology (hetcropolymcrs, 
neural networks) but also in the domain of disordered solids, and in particular spin glasses. Some 
fundamental aspects of these ideas are discussed in an interesting recent paper by Parisi [P]. For 
such systems, many basic problems are not very well understood, and many technical aspects defy 
a mathematical investigation at the present time. An interesting toy model (that nonetheless has 
also practical relevance) where this situation can be studied and for which mathematical results 
are available, is the Hopfield model [FPl,Ho]. This model is a mean field spin system in the sense 
explained above. However, the Hamiltonian, instead of being a function of few macroscopic variables 
is a function of macroscopic variables that are random functions of the microscopic ones and those 
number tends to infinity with the size of the system in a controllable way. More specifically, the 
model is defined as follows. 

Let Sn = {"1)1}''^ denote the set of functions a : {!,..., A'^} {—1,1}, and set S = 
{—1,1}''^. We call a a spin configuration and denote by a, the value of a at i. Let {il,J^,IP) 
be an abstract probability space and let , G IN, denote a family of independent identically 
distributed random variables on this space. For the purposes of this paper we will assume that 
IP[^^ = ±1] = i, but more general distributions can be considered. We will write ^'^[<^] for the 
A^-dimensional random vector whose i-th component is given by S,j^[uj] and call such a vector a 
'pattern'. On the other hand, we use the notation S,i[uj] for the Af-dimensional vector with the 
same components. When we write (^[u] without indices, we frequently will consider it as an M x iV 
matrix and we write S,^[uj] for the transpose of this matrix. Thus, is the M x M matrix 

whose elements are 

Ei=i With this in mind we will use throughout the paper a vector 

notation with (•, •) standing for the scalar product in whatever space the argument may lie. E.g. 



the expression (y, ^j) stands for X^^iCfy^xi 6tc. 

We define random maps m^[a;] : Sn [—1, 1] through^ 

TV 



"^ivM(^) = ^Eer['^K (1-1) 



1=1 



Naturally, these maps 'compare' the configuration a globally to the random configuration ^''[o;]. A 
Hamiltonian is now defined as the simplest negative function of these variables, namely 

iJjvM(a) = -y E (1.2) 

where M(N) is some, generally increasing, function that crucially influences the properties of the 
model. With || • ||2 denoting the ^2-norm in IR^ , (1.2) can be written in the compact form 

N 9 

i7jvM(c7) = --||mjvM(a)||^ (1.3) 

Also, (•, •) will stand throughout for the scaler product of the two argument in whatever space they 
may lie in. 

Through this Hamiltonian we define in a natural way finite volume Gibbs measures on Sn via 

/x^,^M(a) ^ ^—e-'^^-HC-) (1.4) 

and the induced distribution of the overlap parameters 

Qat./sM = MAr,/3M o mAfM"^ (1.5) 
The normalizing factor Z^^js [lj] , given by 

ZnA^] = 2-^ J2 e-^""^'"!^"^ = iE,e-^^^[-l(-) (1.6) 

is called the partition function. 

This model has been studied very heavily in the physics literature. As a basic introduction to 
what is commonly believed about its properties, we refer to the seminal paper by Amit, Gutfreund 
and Sompolinsky [AGS]. Over the last few years, a considerable amount of mathematically rigorous 
results on these measures has been established [BGl,BGPl,BGP2,BGP3,K,N,KP,KPa,ST,PST]. It 
is known that under the hypothesis that limsup^Yi^o M(N)/N = weak limits can be constructed 



^ We will make the dependence of random quantities on the random parameter LO explicit by an added [lo] 
whenever we want to stress it. Otherwise, we will frequently drop the reference to LO to simplify the notation. 



for which the Qn converge to Dirac measures in IR°° [BGPl]. Disjoint weak hmits have also been 
constructed in the case where hmsupjY|^ M{N)/N = a > 0, for small a in [BGP3]. In this note we 
restrict our attention to the case a = and the question to what extent a large deviation principle 
(LDP) for the distribution of the macroscopic overlaps can be proven. 

A first step in this direction had been taken already in [BGP2]. There, a LDP was proven, 
but only under the restrictive assumption M{N) < , while only a weaker result concerning the 
existence of the convex hull of the rate function was proven in the general case a = in a rather 
indirect way. The first LDP in the Hopfield model was proven earlier by Comets [Co] for the case 
of a finite number of patterns. Here we prove a LDP under more natural, and probably optimal, 
assumptions. 

Since the overlap parameters form a vector in a space of unbounded dimension, the most 
natural setting for a LDP is to consider the finite dimensional marginals. Let / C IN be a finite 
set of integers and let IR^ C IR^ denote the corresponding subspacc and finally let 11/ denote 
the canonical projection from IR^ onto IR^ for all p for which / C {1, . . . ,p}. Without loss of 
generality wc can and will assume in the sequel that / = {1, . . . , |/|}. Let us introduce the maps 
^ [-1,1]P through 



where e^, 7 



2" 



7=1 



(1-7) 



1 , . . . , 2^ is some enumeration of all 2^ vectors in JR^ whose components take values 



ibl only. Given I C M, we define the set as the set 

= {me ffil'' I 3y G [-1, +lf'' : n|7|(2/) = m} 



(L8) 



Theorem 1: Assume that lim sup ^ = 0. Then for any finite I C IN and for all < (3 < 00, 
the family of distributions Qn,i3[<^] ° satisfies a LDP for almost all u & Q, with rate function 
Fp given by 



■ sup sup 

Tljnp(y) = m 



2^ "I 

l\My)\\l-p-'2-^Y.^(yy) + (ly'-P-'m) (1-9) 



where 



7(y)^|^Mi + y) + VMi-y) ,^f\y\<l 

[ +00 , otherwise 



(LIO) 



Fp is lower semi- continuous, Lipshitz- continuous on the interior of D\i\, hounded on D\j\ and equal 
to +00 on . 

Remark: Note that Fg is not convex in general. 



To prove Theorem 1 we will define, for m E IR 



(1.11) 



and show that 



i) If m G then 



lim lim 

ejO Nloo 



(1.12) 



almost surely and 



ii) If m G D?^, , then 



lim lim FL f, , (fh) = +oo 



(1.13) 



almost surely. 

Prom these two equations it follows from standard arguments (see e.g. [DZ]) that for almost 
all u for all Borel-sets A C B{JB}) 



The properties of the rate function will be established directly from its explicit form (1.9). 

An important feature is that the rate function is non-random. This means that under the 
conditions of the theorem, the thermodynamics of this disordered system is described in terms of 
completely deterministic potentials. From the thermodynamic point of view discussed above, this is 
an extremely satisfactory result. Namely in these terms it means that although the Hamiltonian of 
our model is a function of an unbounded number of random macroscopic quantities, we may select 
any finite subset of these in which we may be interested and can be assured that there will exist, 
with probability one, in the infinite volume limit, thermodynamic potentials that are functions of 
these variable only and which are, moreover, completely deterministic. The sole condition for this 
to hold is that the number of macroscopic variables goes to infinity with a sublinear rate. 

In the remainder of this article we will present the proof of Theorem 1. There will be three 
important steps. First, we prove large deviation estimates for the mass of small balls in IB!^ ^ 
using fairly standard techniques. The resulting bounds are expressed in terms of a certain random 
function. The crucial step is to show that in a certain strong sense this function is 'self-averaging'. 
The proof of this fact uses the Yurinskii martingale representation and exponential estimates (see 
e.g. [LT]). These are finally combined to obtain deterministic estimates on cylinder events from 
which the convergence result (1.12) then follows rather easily. 




inf ^ Flijn) < limmf - In Q^,,[a;] o Uj' (A) 



(1.14) 



< lim sup — In QnA^] o ^ (^) < - _ inf F^{m) 




2. The basic large deviation estimates 

In this section we recall exponential upper and lower bounds that have already been derived 
in [BGP2]. They provide the starting point of our analysis. 

Let us consider the quantities 



ZN,f3,p[i^]{m) = ^Ar,/3M {\\mN{cr) - m\\2 < p) ^at./jM (2.1) 
We first proof a large deviation upper bound. 

Lemma 2.1: 

-^lnZAr,/3,p(m) <$jv,/3(m)+p(||r||2 + ||m||2 + p/2) (2.2) 



where 



with 



$jv,/3(m)= inf ^NArn,t) (2.3) 



1 1 ^ 

NArn,t) = -{m,t) + - \\m\\l + — Y,lncosh p{Ci,t) (2.4) 

and t* = t*{m) is defined through '^N^fj{m,t*{m)) = infjg^^M ^ N,/3{,fn,t), if such a t* exists, while 
otherwise \\t*\\ = oo. 

Proof: Note that for arbitrary t € IR^ , 

-''■{||mjv(cr)-m||2<p} — ^{\\mNia-)—m\\2<p}^ K'^-'^) 

Thus 



< inf IE e'^^h{\\'^\\l+'^P\\'rr^h+P'^)el^Nit'('iT^N{<7)-rn))+0Np\\t\\2 



(2.6) 



This gives immediately the bound of Lemma 2.1.0 



Remark: Note that if a finite t* (m) exists, then it is the solution of the system of equations 

N 

N 



1 ^ 

m^ = -Y,^^tanhPi^„t) (2.7) 



Next we prove a corresponding lower bound. 



Lemma 2.2: For p> J we have that 



lnZjv,;3,p(m) > ^nA^) - P(Nll2 + \r{rn)h - p/2) - ^ (2.8) 



where the notations are the same as in Lemma 2.1. 



Proof: The technique to prove this bound is the standard one to prove a Cramer-type lower 
bound (see e.g. [Va]). It is of course enough to consider the case where ||t*||2 < oo. We define, for 
t* e IR^, the probabihty measures IP on { — 1,1}^ through their expectation IE(^, given by 

iE^e^^(**'"''^('^))(-) 



We have obviously that 



> -^Ar(t*,m)-/3Ar(p||t*||2-|||m||^+p||m||2-pV2) fSNit* ,mN{cr)) fh -ir^,, , ^ „ ^ ^ 

'^a^ -"^cr-"-{||mAr(<T)— m||2<p} 

X Pa [||"iiv(cr) - m\\2 < p\ 



(2.10) 



But, using Chebychev's inequality, we have that 



-ff'a [||mjv(o") - m\\2 < p] = l- Pa [||mjv(a-) - m||2 > p] 



1 ~ 2 (2-11) 

> 1 ^IEa\\mN{cr) - m\\2 



P' 

We choose t*{m) that satisfies equation (2.7). Then it is easy to compute 

P\\mt,{a)-m\\l = - - tanh2(/?(C„ t*(m)))] (2.12) 

from which the lemma follows. ^ 

In the following lemma we collect a few properties of <I>7v./3 (^n-) that arise from convexity. We set 
r = {m G IR^ I ||t*(m)||2 < oo} where t*(m) is defined in Lemma 2.1, D = {m £ IR^ \ $Ar,^(m) > 
and we denote by intD the interior of D. We moreover denote by I(x) = supjg^ (ta; — In cosh t) 
the Legendre transform of the function In cosh i. A simple computation shows that I{x) coincides 
with the function defined in (1.10). 

Lemma 2.3: 

i) 

1 1 ^ 

where for each m G IR^ the infimum is attained or is +oo vacuously. 

a) 

D = {me I 3y G [-1, l]^ s.t. mN{y) = m} (2.14) 



in) ^N,f3{m) is continuous relative to intD 



iv) r = intD 



v) If t* is defined as in Lemma 2.1 and y* realizes the infimum in (2.13), then 



(2.15) 



Remark: Note that point i) of Lemma 2.3. provides an alternative formula for the variational 
formula (2.3). 

Proof: All results of convex analysis used in this proof can be found in [R]. Note that the func- 
tion g{t) = ^^-^ lncosh/3(.^i, t) is a proper convex function on IR^ . Denoting by h{m) = 
sup(gj^M{(m, t) — g{t)} its Legendre transform, it follows from standard results of convex analysis 
that h(m) is a proper convex function on IR^ and that 



where for each m £ IR the infimum is either attained or is +oo. This immediately yields i). 



dom/i equals the right hand side of (2.14) , and since \\m\\2 > 0, ii) is proven, iii) simply follows 
from the fact that h being convex, it is continuous relative to the interior of dom/i. Finally, to 
prove iv), we will make use of the following two important results of convex analysis. First, the 
subgradient of h at m, dh{m), is a non empty set if and only if m belongs to the interior of dom/i, 
i.e., m G intD. dh{m) is moreover a bounded convex set. Next, {m,t) — g{t) achieves its suprcmum 
at t* = t*{m) if and only if t* G dh{m). To prove v) we only have to consider the case where t* 
exists and consequently \y^\ < 1 for all i. Using the fact that I'{x) = tanh~^(x) and the definition 
of I{x) as the Legendre transform of Incosh(t), formula (2.15) follows from a simple computation. 
This concludes the proof of the lemma. O 

We see that as long as p can be chosen as a function of N that tends to zero as N goes to infinity, 
Lemma 2.1 and Lemma 2.2 seem to provide asymptotically coinciding upper and lower bounds, at 
least for such m for which t* (m) is bounded. The unpleasant feature in these bounds is that 5'jv,/3 
is a rather complicated random function and that the ^n.is is defined through an infimum of such 
a function. In the next section we analyse this problem and show that this function is essentially 
non-random. 




(2.16) 



Denoting by dom/i = |x G IR'^^ \ h{m) < +00} the effective domain of h, we have, by (1.7), that 



3. Self averaging 



We show now that the random upper and lower bounds derived in the last section are actually 
with large probability independent of the realization of the randomness. In fact we will prove this 
under the restriction that m should be such that, at least on a subspace of full measure, t*{m) has a 
uniformly bounded ^2-norm. With this in mind the result will follow from the next proposition. Let 
in the sequel Oi C J2 denote the subspace for which ||?*[u;]^[u;]/iV|| = ||^[u;]^*[u;]/iV|| < (1 + ^)2(1 + 
e) holds for some fixed small e (e = 1 will be a suitable choice). We recall from [ST,BG1,BGP1] 
that IP[ni] > 1 - 4Ne-^^^^\ 

Proposition 3.1: For any R < oo there exists < 5 < 1/2 and a set C f2 with iP[il2] > 
1 - e-^«'"'Vfl^ such that for all w € J^i n ^^2, 



sup \^[Lu]im,t) - IE-^{m,t)\ < a^/^"'^^(6 + 2||m||2) 

t: l|t||2<-R 



(3.1) 



Remark: The subspace does not depend on m. 

Note that an immediate corollary to Proposition 3.1 is that, under its assumptions. 



inf '^\u](m,t) - inf lE^im.t) 

t:||t||2<fl t:||t||2<-R 



< a^/2-^(6 + 2||m||2) 



(3.2) 



Remark: An obvious consequence of (3.2) is the observation that ii m E IR and a; G J^i D ^2 
are such that 

(3.3) 



inf ^'[a;l(m,t)= inf ^\LLi](m,t) 

tgjRM t:||t||2<fl 



and 



then 



inf IE^{m,t)= inf IE^\uj]im,t) 
tenf^ t:||t||2<-R 



$ [w] (m) - inf iE*(m,t) 



(3.4) 
(3.5) 



Proof: The proof of the proposition follows from the fact that for bounded values of t, *(m,t) 
differs uniformly only little from its expectation. This will be proven by first controlling a lattice 
supremum, and then using some a priori Lipshitz bound on ^(m,t). We prove the Lipshitz bound 
first. 

Lemma 3.2: Assume that u e^i. Then 



|*[a;](m,t) - *[a;](m,s)| < {{1 + ^/a){l + e) + \\m\\2) \\t - s\\2 



(3.6) 



Proof: Note that 



\'^{m,t) - •^{m,s)\ < 

< WmhU - sh + 



-{m,t- s) + ^Yl [lncosh(/?(^i,t)) - lncosh(^(^i, s))] 



L ^ [lncosh(/3(e., i)) - lncosh(/3(Ci, s))] 



On the other hand, by the mean-value theorem, there exists i such that 



^^[lncosh(^(Ci,t))-lncosh(/?(^i,s))] = ( t - s, 1 ^ tanh(/3(^i, t)) 



^^(t-s,Ci)tanh(/?(ei,t)) 



Using the Schwartz inequahty, we have that 



% V ^ V ^ 



< 



< 



AT 



\t-s\ 



(3.7) 



(3.8) 



(3.9) 



But this imphes the lemma. <0> 

Let us now introduce a lattice Wn,m with spacing l/\/N in m^. We also denote by Wn,m{R) 



the intersection of this lattice with the ball of radius R. The point is that first, for any t G IR^ , 
there exists a lattice point s G 'Wn,m such that ||s — f||2 < \/a, while on the other hand 



(3.10) 



Lemma 3.3: 



IP 



sup |^'(m,t) - iE'^(m,t)| > re 

teWAr,M(ii) 



< e-^(^(i-5«"^'')-"^"(-^/«)) (3.11) 



Proof: Clearly we only have to prove that for all t G WN,MiR) 



IP[\^{m,t) - IE^{m,t)\ > x] < e-^^^^-'^^^^'"> 



(3.12) 



To do this we write '^{m, t) — IE^{m, t) as a sum of martingale differences and use an exponential 
Markov inequality for martingales. Note first that 

1 ^ 

^{m,t)-IE^{m,t) = — ^(lncosh(/3(^i,t)) -iElncosh(/3(ei,i))) (3.13) 



i=l 



We introduce the decreasing sequence of sigma-algebras !Fk,K that are generated by the random 
variables {Clgg^" U {e^Y^" . We set 



/ff'"^ = IE 



/3-i^lncosh(/3(^i,t))|j^fc,« 



-IE 



where for notational convenience we have set 



•J u 



~ \ if At = M 

Notice that we have the identity 



(3.14) 



(3.15) 



N M 

(fc,K) 

k = l K=l 



(3.16) 



Our aim is to use an exponential Markov inequality for martingales. This requires in particular 
bounds on the conditional Laplace transforms of the martingale differences (see e.g. [LT]). Namely, 
we clearly have that 

N M 



IP 



In 



k = l K=l 



= 2 inf e-l"l^^iB 



> Nx 
IE 



{N M ^ 
k=lK=l ) 



(3.17) 



...IE 



(1,1) 



(1,2) 



N,M 



ueR 

Now notice that 

fi^'''^ = IEy3-'J2^ncoshmi,t))\^k,.] - IE[r^J2lncoshm,,t))\K^] 

i i 

= IE[p-' In cosh(/?(a , m^k,.] -mr' In cosh(/3(a , *)) 1^^+ J 

= IE[/3-' lncosh(/3 I ^ C^t^ + ^^t^ j )\J^k,.] - mP'' lncosh(/3 I ^ ^^t^ + ^^t^ ) 



lncosh(/3 I i'A + St. ) - liiC05h(/J Y. ^''f - I )|^t,« 



Now we use the fact that 



cosh (a + 6) _ l + tanhatanh6 ^ l + tanh|6| ^ 2\h\ 



cosh(a — h) 1 — tanh a tanh h 1 — tanh |6| 



(3.18) 
(3.19) 



to deduce from (3.18) that 

< \t.\ (3.20) 
Using the standard inequaUties < 1 + a; + ^el^' and 1 + y < we get therefore 

,2 



IE 

Prom this and (3.17) we get now 



<exp{^t^,eHI*''l) (3.21) 



IP[\^{m,t)-IE^{m,t)\ >x]< 2inf e-'^^^+^^ll*"^'^"''"*"^ 

u 

( _jv^ri-ie"^/ii*ii2) (3.22) 
< J 2e ' ifp||2>l 

\2e-^x^(i-K), if ||t||2<l 

where the last inequality is obtained by choosing u = a;/||i||2 in the first and u = x/\\t\\2 in the 
second case. This gives the lemma. 

We can now continue the proof of Proposition 3.1. Choose < S < 1/2 and define ^2 to be 
the set of a; G n for which 

sup \^{m, t) - IE^{m, t)\ < a^l'^-^ (3.23) 
teWiv,M(R) 

By Lemma 3.3, 

.,1-2(5 



JP\^2\ > 1 - exp f-iV^^(l - -e°'^'""/^) + iValn(i?/a; 
\ R 2 

= 1 - exp {-NO{a^-'^^ IR)) 



(3.24) 



Combining Lemma 3.2 with (3.23) and taking into account the remark preceeding Lemma 3.3, we 
see that on $7i n il2, 

sup \^{m,t)-lE-^{m,t)\ < a^/^"'' + 2Va(||m||2 + (1 + Va)(l + e)) < a^/2-'^(6+ ||m||2) (3.25) 

t :||t||2<-R 

for a small, which proves Proposition 3.1.<0' 



4. Proof of the Theorem 



The results of Sections 2.1 and 3.1 can now be combined to get a large deviation principle in 
the product topology. The point here is that, apart from the possibility that t*{m) may become 
unbounded, the estimates in Lemma 2.1 and Lemma 2.2 together with Proposition 3.1 tell us that 
up to corrections that tend to zero with iV, the quantity {[3N)~^ InZj^ ^ ^^i''^) is given by the 
infimum over t of the completely non-random function lE'^ N,j3{m,t). We will first prove that for 
all m G Di (1.12) holds. The main step in the proof of this fact is the following theorem. 

Theorem 4.1: Assume that limsup^Yi-oo ~~w^ = '^'^^ ^^^^ < /3 < oo. Then there exists a 
set Cl C 0, with IP[Cl\ = 1 such that for all finite subsets I C IN and for all m [—1,1]^ such that 
for all e > there exists c = c{m, e) < oo, 3A^o ^ oo,yN > Nq, 



sup inf IE^N,/3{m,t) = sup inf IE'^N^/3{m,t) (4.1) 

m: ||njm— m||2<e m: ||njm— rh||2<e ^^-'^ • ll*l|2<c 



it holds that for all to E 

lim lim FL q [u] im) 



sup sup 

pew ^6[_i,i]2P 

n/np(ii) = m 



1„ 



2P 



l-p-'2-^^I{y, 

7=1 



+ SUP (L^-p-^I(y)] 



(4.2) 



Remark: The assumption in Theorem 4.1 looks horrible at first glance. The reader will observe 
that it is made in order to allow us to apply the self-averaging results from the last section. We will 
show later, however, that the set of values fh for which it is satisfied can be constructed explicitly 
and is nothing else than D^j^. 

Proof: We will first establish an upper bound for the quantity 

■^Ar,/3,eM("^) = m,/3[w] (||n7mAr(o-) - m\\2 < e) ^Ar,/3[w] (4.3) 

To do so, notice that on ||mAr((7)||2 < (1 + (1 + e) < 2 for all a. We may cover the ball 

of radius 2 with balls of radius p ~ ■s/a, centered at the lattice points in Wn,m{'2')- We then have 
that on 

IIITjm — m||2<e 

< sup ZN,i3,p[^]i^) ^ r4 4) 

" ^ ||njm-m||2<e 

< sup Z^,^,p[a;](m)e«^(i"2/a) 

m: ||m||2<2 
lirirm — m||o<e 



As long as a 1 0, the factor e"-'^('"^/") in the upper bound is irrelevant for the exponential asymp- 
totic, as is the difference between e and e — p. Using the estimates used in the proof of Lemma 2.1, 
we can replace Zjv,/3,p[a;](m) in (4.4) by its upper bound in terms of the function ^. Namely, 

-^\nZl^p[u]{m)< sup inf^jv[u;](m,i)+p(c + 2 + p/2) + /?-ialnl/a 

m:||m!|2<2 teJR*^ K^''-' ) 

||n/m-m||2<e I|t|l2<° 

Finally, combining (4.5) with (3.2), we get that, for w G rii n and for any c, 

-J— In Z If gjuj]{m) < sup inf iE*jv(m, t) + lOa^/^"*^ + p(c + 2 + p/2) + /J-^a In l/a 

PJ^ m: ||njm-7n||2<e jeK*^ 

~ ll'IbSc 

(4.6) 

By assumption, there exists a value c < oo, such that the true minimax over lE'^ pj{m,t) is taken 
for a value of t with norm bounded uniformly in N by some constant c. The constant c in (4.6) 
is then chosen as this same constant, and then the restriction ||t||2 < c is actually void, and the 
minimax is taken for some values {m* ,t*) which depend only on rfi and e. This is already essentially 
the desired form of the upper bound. 

We now turn to the more subtle problem of obtaining the corresponding form of the lower 
bound. Trivially, 

^^,/3,e+pM(^) > ^iV,/3,pM(m*) (4.7) 

We will modify slightly the derivation of the lower bound for ZM,i3,p[oj\{m*). Namely, instead of 
defining the shifted measure P with respect to the random value of t that realizes the infimum of 
*jv[^^]('7i*,i), we do this with the deterministic value t* that realizes the infimum of lE'^ ^{m* ,t). 
This changes nothing in the computations in (2.10) and (2.11). What changes, is however the 
estimate on ^o-||'7iAr(c) — since t* does not satisfy (2.7) but is instead solution of the 

equations 

m* =iEertanh(^(6,r)) (4.8) 

Thus in place of (2.12) we get 

^a\\fnN{(^) — '"T'*||2 = 



a=iCOsh/3(ei,t*) 
= jpT.Y.^ + jpY.Y. tanh(/3(r ,0)) tanh(/?(t* , a))?^^ 



(4.9) 



2^ E E < tanh(/?(r , 0))CJ + Y.^mlf 

j U V 



N \ N 



The first summand in (4.7) is bounded by a, and we have to control the second. To do so we use 
(4.8) to write 

= Y.(^J2^i tanh(/3(r ,^,))-IE^J2^^ tanh(/3(i* ,^,))\ 

u \ i i / 

= GNin 

We will now prove, in analogy to Proposition 3.1, that G]sf{t) is actually small with large probability. 
This will be slightly more complicated than in Proposition 3.1 and will, in fact consist of two steps. 
The first is a fairly crude bound on GAr(i) that in a second step will be used to obtain a refined 
one. 

Lemma 4.2: For all a; G Oi, 

GivM(t) < 6 (4.11) 



Proof: Let us for notational simplicity set Tj = tanh(/5(^j, i)). We have that 



M 



2 

iV2 



M 



(4.12) 



For the first term, we can use simply that 



M 



N 



< 2 



N 



(4.13) 



But on Jli, the norm in the last fine is bounded by (1 + ^/a)^{l + e). To bound the second term in 
(4.12), we use the independence of both and Tj for different indices i to write 

M M 
At=l i,j n=l i,j 



M 



(4.14) 



< 2IE 



N 



+ 



2M 



N 



< 2a + 2(l + V^)2(l + e') 



Combining these two bounds we get (4.11).>s) 

Lemma 4.2 tells us that G'jv(t) is bounded, but not yet that it is small. To do this, we observe 
first that its mean value is small. 

Lemma 4.3: 

< IEGN{t) < a (4.15) 



Proof: 



M 



IEGN{t) = J2^ 

M I 



I/=l 



l^etanh(/3(f,C,)) - l^iECrtanh(/3(f,C,)) 



v=l \ i i / 



(4.16) 



< 



M 



where we have used the independence of the summands for different indices i. 

In the sequel wc will need that the mean value of Gat (i) does not differ much from its conditional 
expectation relative to Oi. Namely, 



\IEGN{t) - iE[Gjv(t)|f^i]| < 2Me 



-eAri/6 



(4.17) 



is arbitrarily small. 



Finally, wc will show that on with large probability, 0^(1) differs only little from its 
conditional expectation relative to Oi. 



Lemma 4.4: Assume that x » {In N)/^/N. Then, 



IP [\GN{t) - IE[GN{t)\ni]\ > x\ni] < e-''^^ 



(4.18) 



for some positive constant h. 



Proof: Basically the proof of this lemma relies on the same technique as that of Proposition 3.1. 
However, a number of details are modified. In particular, we use a coarser filtration of !F to define 
our martingale differences. Namely, we denote by Tk the sigma algebra generated by the random 



variables {Cf }f>f^- We also introduce the trace sigma algebra ^ = JT n Hi and by JT^ = JT^ PI Qi 
the corresponding filtration of the trace sigma algebra. We set 



(fc) 

AT 



IE 



IE 



GN{t)\Tk+\ 



Obviously, we have for a; G fii 



N 



GN[u]{t)-IE[GN{mi] = Y,f^ 



(fc) 

N 



(4.19) 



(4.20) 



fc=i 



IE 



Ak) ^ 



fe+i 



The strategy to get those is very similar to the one used in [BGP3] and [B] . We introduce 

fj, \ iy^k i J 



Thus the lemma will be proven if we can prove an estimate of the form (4.18) for the sum of 
the Z^-*. This goes just as in the proof of Proposition 3.1, i.e. relies on uniform bounds on the 
conditional Laplace transforms 

(4.21) 

(4.22) 

(4.23) 
(4.24) 

(4.25) 
(4.26) 



and set 



We then have that 



since G^''(t,0) is independent of the random variables ^fe. On the other hand. 



= IE 



5fc(l)l-^fe 



-IE 



9k{i)\^k+i 



and 



M 



9k 



u^) = 2E 



I/=l 



gk{l) = / dzg'kiz) 
Jo 

T7 E ^i'^^ ~ E ^i'^i + l^^kTk 



Ak) 



J^^k'^k 



Let us first get a uniform bound on |/)^r | on Q^. From the formulas above it follows clearly that 



|/(f)|<2sup|5U^) 



(4.27) 



But using the Schwartz inequality, 



\9'ki^)\ < j^i: 



<-Vm 



i^k 



E 



(4.28) 



2VM 



\fG^. 



But on rii it is trivial to check that G^^\t, z) satisfies, for z G [0, 1], the same bound as GN{t). So 
that on Jli, 



12VM 

N 



(4.29) 



Now we turn to the estimation of the conditional Laplace transform. Using the standard inequality 



we get 



IE 



< 1 + -u'^IE 



(/, 



(fc) 

N 



(fc) 



A simple computation (see [BGP3]) shows that 



IE 



< 4:IE 

= AIE 

< AIE 



(5fc(l)) 



< 4 sup iE 

0<z<l 



fc+1 



I dz {9k{z)f \^k+i 
Jo 

{g',{z)f \^k+i 



Let us write 



M 



9k 



'kiz)=2Y^ 



M 



N 



CkTk 



Thus 



M 



(5U^)) <8 E 



^=1 



+^n{z-i) 



iV4 



(4.30) 



(4.31) 



(4.32) 



(4.33) 



(4.34) 



Let us abbreviate the two summands in (4.34) by (I) and (II). The term (II) is of order a^A~^ and 
thus can simply be bounded uniformly. We have to work a little more to control the conditional 
expectation of the first. We write 



IE 



k+l 



N2 



IE 



^■k^k-'-k 



(4.35) 



We observe that under the expectation conditioned on J^k+i we may interchange the indices of 
1 < i < and use this to symmetrize the expression (4.35). (Notice that this is the reason why we 
separated the z-dependent contribution in (4.34)).This gives 



IE 



IE 



Ar2 



< -^lE 

- 



k pi^cv 



ii,v j=l 



k c ct 

k ' 



M 

E 



i i J 



N 

2 



J", 



k + 1 



^k+\ 



But by Lemma 4.2, on 

M 

Y. 

and since 



Giv(i) < 6 



< 



k 



we get that 



IE 



1 48 48 

< [ii5(fc)ii < ^iE\\Bm/ipm 



7V2- 



(4.36) 



(4.37) 



(4.38) 



(4.39) 



It is easy to show that (see [B]) that 

IE\\B{k)\\ <c(l + 



(4.40) 



for some constant 2 > c > 1. Collecting our estimates and using that 1 + x < we arrive at 



IE 



e^'f-'l^k+i < exp ( ^u2eMi2VM/iv^-24 Sa^ + 76{1 + ./M/kf 



1 



(4.41) 



Since 



N 



^(1 + ^M/kf = N + 4VMN + MlnN = N(l + + alnN) (4.42) 



IP 



fc=i 

this yields that 

N 

,{k) 

lk=l 

In order to perform the infimum over u in (4.43) we must distinguish two cases. First, if a < 1/ In A^, 
we may chose u = a/JV which yields 

N 



< inf exp (^-ux + ^e^^^^'^^'^ 4. [Sa^ + 76 + 304v^ + 76a In AT] ^ (4.43) 



IP 



.k=l 



^ g-\/iVx+ci 



(4.44) 



for some positive constant ci. If now a goes to zero with N more slowly than 1/lnA?^, a good 
estimate of the infimum is obtained by choosing u = N/12-\/M. This gives 



IP 



N 



.fc=l 



< e i2v^ exp - 

^ -y/Nx/12+C2 InJV 



12 48 

\- — +2lnN 

a y a 



(4.45) 



for some positive constant C2. From here the lemma follows immediately. <0> 

Corollary 4.5: There exists a set Cls{t*) C Hi with IP['D,i\U3] < e~^^^^^ such that for all 

1 



IPn 



\mN{a) - m* \\^ < [2(2a + AT" V4)]i/2 



> 



(4.46) 



Proof: This follows from combining (4.9) and (4.10) with Lemmas 4.2, 4.3 and 4.4 and choosing 
X = Af-V4 in the latter.O 

Since by assumption ||t*||2 < c, lemma 2.3 implies that on Cli, ||m*||2 < 2. As a consequence, 
putting together Proposition 3.1, Corollary 4.5 and (2.10), we find that on (t*), 

-i^lnZ^,^,,+^[u;](m) > IE^N{m*X) - Wa'/'-' - p{c + 2 - p/2) - ^ (4.47) 

Which is the desired form of the lower bound. 

Finally, by a simple Borel-Cantclli argument, it follows from the estimates on the probabilities 
of the sets ^i, 0,2 and 0,3(t*) that there exits a set of measure one on which 

lim sup In g [w] (m) < lim sup sup inf IE^N{m,t) (4.48) 

and 

liminf — — InZL ,[a;l(7fi) > liminf sup inf IE^N(m,t) (4.49) 

JVToo PN '^'^'^^ ATToo m:\\njm-rhh<e-ptem^ 

It remains to show that the limsup and the liminf's on the right-hand sides of (4.48) and (4.49) 
coincide. Prom here on there is no difference to the procedure in the case M < In N/ In 2 that was 
treated in [BGP2]. We repeat the outline for the convenience of the reader. We write IE"^N{m,t) 
in its explicit form as 

1 ^ 

IE<lfN{m,t) = - \\m\\l - (m,t) +/3-^2-^^lncosh(/3(e^,t)) (4.50) 

7=1 

where the vectors e^, 7 = 1, . . . , 2^ form a complete enumeration of all vectors with components 
±1 in m^. They can be conveniently chosen as 



= (_1)[72^-1 (4.51) 



where [x] denotes the smaller integer greater or equal to x. Note that IE'^N{m,t) depends on N 
only through M{N). We may use Lemma 2.3 to show that 



inf iE*jv(m,t) = -||m||^- inf (3-^2-^ I{y-y) (4.52) 

and hence 

2M 

sup mi^IE^N{m,t)= sup l\\np{y)\\l - ^-^2-"^ ^I{y^) (4.53) 

m: ||njm-m||2<e*eiR yelR'^'^ : \\ninp{y)-rh\\^<e 7=1 



To prove that this expression converges as A'' (or rather M) tends to infinity, we define the sets 

Af = [ye[-l,l]^" \y^ = y^+2^ } (4.29) 

Obviously, 

< C < C . . . A^_, CA^ = [-1, 1]^" (4.30) 
The definition of these sets implies the following fact: If y G with d < M, then 

(i) np{y) = 0, if u > d and 

(ii) n^(y) = n'^{y), li ii<d. 
Let us set 

7=1 

and 

Tp,e(m)= sup Qp{y) (4.33) 

yeA^ 

l|njnp(y)— T7j||2<e 

Therefore, for y G 0p(y) = @d{y), while at the same time the constraint in the sup is satisfied 
simultaneously w.r.t. Up or na, as soon as d is large enough such that I C {I, . . . ,d}. Therefore, 



Tp,e(m)> sup Qp{y)= sup @d{y) = Td,e{rri) (4.34) 
yeA¥ yeA'' 

a a 

l|n/rjp(j;)— m||2<e II n j rjp (y) — m || 2 <« 



Hence Tp^^(m) is an increasing sequence in M and being bounded from above, converges. Thus 

lim sup inf IE'^N(m,t) = lim T„g(m) 

= sup Tp^^{m) 
p 



It remains to consider the hmit e J, 0. It is clear that sup^ Tp^g(m) converges to a lower-semicontinuous 
function and that 

Hmsup Tp^e(m) = Hm sup supTp^o("^) (4.55) 

p ' '^l^ m:\\Tlim-rh\\2<e p 

Thus if supp Tp^o("^) is continuous in a neighborhood of rh, we get 

limsupTp^e(77i) = supTp^oC?^) (4.56) 

elO p ' P ' 

as desired. But, as has been shown in [BGP2], from the exphcit form of T one shows easily that 
supp Tp^o("^) is Lipshitz continuous in the interior of the set on which it is bounded. This proves 
Theorem 4.1 

We will show next that a sufficient condition for condition (4.1) to hold is that m belongs to 
-D|7|. While this appears intuitively 'clear', the rigorous proof is surprisingly tedious. Let us first 
introduce some notation and results. 

Let Ep be the p x 2^'-matrix whose rows are given by the vectors e^, 7 = 1, . . . , 2^, which, 
for convenience, are ordered accordingly to (4.51). We will denote hy , = 1, . . . ,p the column 
vectors of Ep and by Ep its transpose. It can easily be verified that 

2-P(e'^,en = I ^ if ^ = z^ 

^ ' ' lO otherwise ^ ^ 

Thus, the 2^ x 2P-matrix 2~PEpEp is a projector that projects on the subspace spanned by the 
orthogonal vectors {e'^}^_]^, and 2~PEpEp is the identity in lEF. Given a linear transformation A 
from RP to R'^, we define 

AC = {Ax\xe C} for C C mF (4.58) 
With this notations the vector np{y) and the set Dp, defined in (1.7) and (1.8), can be rewritten as 

np{y) = 2-PEly 

(4-59) 

Dp = 2-PEl[-l,ir 
Moreover, for any set / C {1, . . . ,p}, we have the following property, 

UiDp = D^j^ (4.60) 



Finally, let us remark that of course the statements of Lemma 2.3 apply also to the deterministic 
function inf^giRP lE'^ i^^p{m,t) . All references to Lemma 2.3 in the sequel are to be understood as 
referring to properties of this latter function. 

By Lemma 2.3, the condition (4.1) of Theorem 4.1 is satisfied if and only if the supremum in 
the l.h.s of (4.1) is taken on at a point m in \rADp. More precisely, by (2.15), this condition is 



equivalent to demanding that for all e > and all p, the supremum over y s.t. ||n/np(y) — m\\2 < e 
of 6p(y) is taken on at a point y* such that 

2-"E[^'(y;)]'^^ (4-61) 

7=1 

We set 

Aeim) = [ye [-1, if"" : \\Uinp{y) - rhh < e} (4.62) 



Lemma 4.6: Assume that < /? < oo. Then for all m G -D|7| and e > there exists c{rh, e) < oo 
such that for all p > \I\ 

inf e{y) = e{y*) (4.63) 

njnp(3/)€Be(m) 



where 



r,(y*) = 2-^5;[/'(y;)]'<c(m,6) (4.64) 

7=1 



Proof: The proof proceeds by showing that if y does not satisfy condition (4.64), then we can find 
a Sy such that y + 6y^ Ae{rh) and &p{y + Sy) < Qp{y), so that y cannot be the desired y*. Let us 
first note that 

1 

Qpiy + Sy) - Gp(2/) = - [||np(2/ + 6y)\\l - \Kiy)\\l] + ' E [^(2/7) " ^(2/7 + '^^7)] (4-65) 

^ 7=1 

Using the properties of the matrix Ep we can bound the difference of the quadratic terms as follows 



Kiy + Sy)\\l - \\np{y)\\l =\\npi5y)\\l + 2-P+\5y, 2-PEpE^y) 

>_2-p/2+i||<5y||2 



(4.66) 



Thus we can show that @p{y + 6y) < @p{y) holds by showing that 

2" 



^ [/(y^) _ /(y^ + ^y^)] > 2-P/^Sy\\2 (4.67) 

7=1 



Define the map Y from [—1, l]^'' to [—1, 1]^'^' by 



2P-\'\_i 

K,(y) = 2-^'+l^l J2 2/7+72m > 7 = I,-- -,21^1 (4.68) 

7=0 



Using (4.59) we get that 

(2P-\'\ 
2-v+\i\ n^(^_i)2m+i„.„^2m}2/ | = 2-\'\ El^Y {y) (4.69) 
7=1 

Therefore, the property that y G Ae{fn) depends only on the quantity Y{y). 

Notice that if m G D\j\ and e > 0, then there exists X G (—1, 1)^'^' such that ||n7(X)— ?fi||2 < e. 
This imphes that for any p, the vector x G IR^'^ with components = -'^j^mod2i^i li^s also in Ae{rh). 
Moreover, 

max |a;-y| = max = 1 — d < 1 (4.70) 

7 7 

and therefore 

T^{x) < [I'il - d)f (4.71) 

is some finite p-independcnt constant. We will use this fact to construct our 6y. We may of course 
choose an optimal X, i.e. one for which d is maximal. In the sequel X and x will refer to this 
vector. Let now y be a vector in Ae{rh) for which Tp{y) > c for some large constant c. We will 
show that this cannot minimize @p. We will distinguish two cases: 

Case 1: Let us introduce two parameters, < r) <^ d and < A < 1, that will be appropriately 
chosen later. In this first case we assume that y is such that 

El{l..l>i-'7}> (1-^)2^"'" (4-72) 

7=1 

and we choose 

5y = p{x - y) (4.73) 

where < p < 1 will be determined later. It then trivially follows from the definition of x and the 
convexity of the set Ae that y + p{x — y)^Ae and that y + p{x — y) G [— 1 + pd, 1 — pd\^^ . If Thus 
if we can show that with this choice, and with an p such that pd > r], (4.67) holds, we can exclude 
that the infimum is taken on for such an y. 

Let us first consider components such that \y^\ > 1 — d. Since \x^\ < 1 — d we have, for 
those components, sign5y^ = — signj/-,, and thus liy-y) — I{{y + Sy)-y) > 0. This fact together with 
(4.72) entails 

2" 2" 

2-"" Y^i^ivj) - Hiy + (5y)7)]i{|,,i>i-d} >2-^ Y^i^ivy) - Hiy + (^y)7)]i{|y,i>i-,} 

7=1 7=1 (4.74) 

> inf (l-A)2-l^l[/(y^)-/((y + <5y)^)] 

ly-f I >i— a 

{X'y I < 1 — d 



Note that I{z) is symmetric with respect to zero and is a strictly increasing function of z for z > 0. 
Thus I{{y + Sy)^) is maximized over < 1 — d for = (1 — (i)sign?/^. Prom this we get 

|a;-y |< 1— a 

and the infimum over \yj\ > 1 — 77 in the r.h.s. of (4.75) is easily seen to be taken on for \y^\ = l — rj. 
Thus 

inf (1 - A)2-I^l [I{y^) - I{{y + 5y)^)] >(1 - A)2-I^l [1(1 - r?) - 1(1 - ,? - p{d - ry))] 

li^-y I>1 — a 
I a;-y I < 1 — d 

>(1 - A)2-l^l/9(d - ??)/'(l -7]- p{d- 7])) (4-76) 
>(1 - A)2-I^lp(d - ry)i| ln(r/ + p(d - rj))\ 

where we have used the convexity of / and the bound, — x) > || lna;| for < x < 1. 

We now have to consider the components y-y with \y-y\ < 1 — d. Here the entropy difference 
liy-y) — I{{y + Sy)'y) can of course be negative. To get a lower bound on this difference we use (4.75) 
and perform the change of variable \yj\ = {1 — d) — Zj to write 

, [^(y7)-^((y + W]=,^inf I{{l-d)-z^+pz^)-m-d)-z^) 

Ix-y I < 1 — d 

0<z-y<l— a 

1, f 2 — d — Z-y + PZ-y \ 

= mf -02;^,- In — ; ^ — ^— i (4.77) 

0<z-y<l-d '2 V d + Zj—pz-y ) ^ ' 

>-p(l-<i)iln(^ 

-2d 

and putting together (4.77) and (4.72) yields 

2-^ Y^AV.) - Hiy + ^yhM{M<i-d} > -(l - (l - A)2-I^I)| in (4.78) 

7=1 ^ ' 

Therefore, (4.78) together with (4.74) and (4.76) give 

r'2-^Y.i^iy,)-I{iy + 6y),)] 

7=1 (4.79) 

>r V |(1 - A)2-l^l id - v)l\Hv + P{d - r?))| - (1 - (1 - A)2-I^l)i In 0' 
On the other hand, we have 

2-^/'||<5y||2<2p (4.80) 



Consequently, (4.67) holds if we can choose A, 77, and p so that the following inequality holds, 

(5-' |(1 - A)2-I^l(d - r/)i| ln(r/ + p{d - 77))! - (1 - (1 - A)2-I^l)i In (^) } > 2 (4.81) 

But this is always possible by taking e.g. A < 1, 77 = pd/2 and p = where K = K[d, A) > 1 
is chosen sufficiently large as to satisfy 

(1- A)2- 1-^1^(1 -d^/2);^i^ I In d| >4+^|lnd| (4.82) 

Case 2: We will assume that A < 1, and that rj, and p are chosen as in the case 1. We can then 
assume that 

7=1 

We assume further that 

Tp(y) > c (4.84) 
for c sufficiently large to be chosen later. Here we will choose 6y such that 

Y{6y) = (4.85) 

so that trivially y + 6y £ Ae{rh). Let us introduce a parameter < < ij, that we will choose 
appropriately later, and let us set, for 7 G {1, . . . , 2l^l}, 

/C+ = {7 G {1, . . . , 2P-\'\} I |y^+(^_i)2m I > 1 - a (4.86) 

and 

K- = {je{l,..., 2P-\'\} I |y^+(;^_i)2m I < 1 - r/} 

For all indices 7 such that /C+ = 0, we simply set (5?/^_|_(^_i)2i/| = for all 7 G {!,..., 2^'^-^^}. If 
/C+ were empty for all 7, then Tp{y) < [/'(I — (^)]^ which contradicts our assumption (4.84), for 
suitably large c (depending only on Q. Thus we consider now the remaining indices 7 for which 

First note that (4.83) implies that |/C+| < (1-A)2p-1-^I and that /C" > A2p-I^I so that choosing 
1 > A > ^, we have |/C^| < |/C~|. Our strategy will be to find Sy in such a way as to decrease the 
moduli of the components in at the expense of possibly increasing them on /C~ in such a way 
as to leave Y{y + 5y) = Y{y). 

We will in the sequel consider the case where there is only one index 7, e.g. 7 = 1, for which 
/C+ is nonempty. The general case is treated essentially by iterating the same procedure. We will 
use the simplified notation yij^2\'\^ = Vii ^2/1+21^17 = ^V'y ^^id also set /Cf = /C^. We will assume 



moreover that all components yx^ are positive, as this is the worst situation. We will chose 5y such 
that 5y^ = 0, if 7 G {/C+ U 1C~Y and 5y^ < if 7 G /C+. For each 7 G /C+ we will choose in a 
unique and distinct 7' G /C~ and set = —Sy^. This ensures that Y{Sy) = 0. We will also make 
sure that for all 7, \dyj\ < •q/2 — (. 

We have to construct 6y^ for 7 G /C"*". In this process we have to consider the following three 
functionals: 

(1) The change in the quadratic term of @p. This is bounded by 



6E{dy)^2-P/'+^ 2 5y? (4.87) 

(2) The change in the entropy term, 

SI{6y) = 2-^ ^ {I{y~+6y^)-I{y^)) 

+ 2-^ J2 il{y^ + 6y^)-l{y^)) 
T-e/c- 

> 2-P \Syi\l'iyi + Sy^) - E ^y^\ 1^^1/2 (4.88) 

= 2-'' E l'Jy7l(^'(y7 + '^y7)-|in^l/2) 

> 2-^-1 E l^y7l^'(y7 + <^2/7) 

7e/c+ 

where we have used that for 1 > |x| 3> \y\ ^ 0.9, I{x) — I{y) ^ \x — j/||ln(l — y)\ and that 
under our assumption for 7 G /C"*", y^y + 5y^ > 1 — r]/2. 

(3) Finally, we have that 

Tp{y + 5y) < 2-f E i^'iVi + <^2/7)]' + 2"^ E [^'(^^^ + <^2/7)]' 

(4.89) 

<[/'(l-r?/2)]2+2-^ E V\Vi + ^y^'^^ 



Looking at these three functionals suggests to choose for 7 G /C+ as the solution of the 
equation 

-5y^ = Tl'{y^ + 5y^) (4.90) 
The point is that with this choice (4.89) yields (we set for simplicity 5E{5y{T)) = SE{t), etc.) 

5I{t) > ^{6E{T)f (4.91) 



while 

Tp(r) < [7'(1 - C)]' + r-\dE{T))' (4.92) 

Thus we can ensure that the entropy gain dominates the potential loss in the quadratic term 
provided we can choose r < 5E{t)/S. However, we know that Tp(t) is a continuous function of t 
and TpijS) > c. Thus there exists tq > such that for all r < tq, Tpij) > c/2, and so by (4.92), 



t-HE{t) > Vc/2-[/'(l-C)P (4.93) 

which inserted in (4.92) yields that 

51 {t) > ^I^^c/2-[I'il-0?SE{T) (4.94) 

It is clear that if c is chosen large enough ('large' depending only on Q, this gives SI{t) > 6E{t), as 
desired. Finally, it is easy to see that \Syj\ is bounded from above by the solution of the equation 

X = tI'{1 - x) (4.95) 

which is of the order of x ~ r| Inrj. If C is chosen e.g. ( = rj/i, we see from this that for small 
enough r, < 77/2 — (, so that all our conditions can be satisfied. Thus, there exist c < 00 

depending only on r] (which in turn depends only on m and e) such that any y that satisfies the 
assumptions of Case 2 with this choice of c in (4.84) cannot realize the infimum of Op. The two 
cases combined prove the lemma. <0 

To conclude the proof of Theorem 1 we show that for -m e Dj (1.13) holds. This turns out to 
be rather simple. The main idea is that if m G D^j^ , then on a subset of Q of probability one, for 
N large enough and e small enough, the set {a G Sn \ \\^i'mN{a) — rh\\2 < e} is empty. 

To do so we will first show that uniformly in the configurations a, the vector UiniNio') can be 
rewritten as the sum of a vector in D\i\ and a vector whose norm goes to zero as N goes to infinity. 
Let e-y, 7 = 1, . . . , 2l^l , be the column vectors of the matrix E^j^ . We set 

t;^ = (i G {1, . . . ,iV} I = ef; , V/x G /} (4.96) 

These sets are random sets, depending on the realization of the random variables . Their cardi- 
nality, however, remains very close to their mean value. More precisely let A-y denote the fluctuation 
of l^-yl about its mean. 



2-1^1 iV 



(4.97) 



There exists a subset i}^ € il. of probability one and a function, 6n, tending to zero as N tends to 
infinity, such that for all but a finite number of indices, 

|A^| <<5^ , 7=l,...,2l^l (4.98) 



This fact has been proven in [G]. Using (4.96), Il/mjvCo") can be rewritten as 



UiruNia) = 2-^^^E^j^{X{a)+5X{a)) (4.99) 

where X{a) and {6X){a) are respectively the vectors with components Xy{a) = \v-y\~^ Sie-y^ ^ 
[— 1, 1], {SX)^{a) = X^X^{a), 7 = 1,..., 21^'. It then foUows from the properties of the matrix E^^j^ 
and (4.98) that, on ^^4, 

\\UjmN{cT) - n,,,(X(a))||2 < Sn (4.100) 

Now, by assumption, rh G D^i\, i-e. there exists e > such that {x G iZ?'^' | \\x — rh\\2 < e} C 
{DiiiY. Therefore, since n\ii{X{a)) G we have ||n|7|(X((T)) — m\\2 > e. Prom this and (4.100) 
it follows that on O4, \\IljmN{a) — rh\\2 > e — Sn. Finally, for N large enough and e small enough 
we get 

{c7 G <Sjv I ||n/mjv(cr) - rhy < e} = (4.101) 
From this, part (1-13) easily follows. This concludes the proof of Theorem 1. 
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